{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "fca96fe5",
   "metadata": {},
   "source": [
    "# Sklearn中逻辑回归建模"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "405ab30b",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-23T09:17:25.978967Z",
     "start_time": "2023-05-23T09:17:25.963587Z"
    }
   },
   "source": [
    "## 分类模型的评估"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b77a0a06",
   "metadata": {},
   "source": [
    "在前面内容中，讲解了回归模型的评估方法，主要有均方误差MSE，R方得分等指标，在分类模型中，我们主要应用的是准确率这个评估指标，除此之外，常用的二分类模型的模型评估指标还有召回率（Recall）、F1指标（F1-Score）等等。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90d87c3c",
   "metadata": {},
   "source": [
    "### 准确率的局限性"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62bb8dff",
   "metadata": {},
   "source": [
    "准确率的定义是：对于给定的测试集，分类模型正确分类的样本数与总样本数之比。举个例子来讲，有一个简单的二分类模型model，专门用于分类动物，在某个测试集中，有30个猫+70个狗，这个二分类模型在对这个测试集进行分类的时候，得出该数据集有40个猫（包括正确分类的25个猫和错误分类的15个狗）和60个狗（包括正确分类的55个狗和错误分类的5个猫猫）。画成矩阵图表示，结果就非常清晰\n",
    "\n",
    "<img src=\"./images/08.jpg\" alt=\"1\" />\n",
    "\n",
    "从图中可以看出，行表示该测试集中实际的类别，比如猫类一共有25+5=30个，狗狗类有15+55=70个。其中被分类模型正确分类的是该表格的对角线所在的数字。在sklearn中，这样一个表格被命名为混淆矩阵（Confusion Matrix），所以，按照准确率的定义，可以计算出该分类模型在测试集上的准确率为："
   ]
  },
  {
   "cell_type": "markdown",
   "id": "659af0bc",
   "metadata": {},
   "source": [
    "$$\n",
    "Accuracy = \\frac{25+55}{25+5+15+55} * 100\n",
    "$$\n",
    "\n",
    "即，该分类模型在测试集上的准确率为80%"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7bbc5b9",
   "metadata": {},
   "source": [
    "在分类模型中可以定义\n",
    "- Actual condition positive（P）：样本中阳性样本总数，一般也就是真实标签为1的样本总数；\n",
    "- Actual condition negative（N）：样本中阴性样本总数，一般也就是真实标签为0的样本总数；\n",
    "- Predicted condition positive（PP）：预测中阳性样本总数，一般也就是预测标签为1的样本总数；\n",
    "- Predicted condition negative（PN）：预测中阴性样本总数，一般也就是预测标签为0的样本总数；"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5937c70a",
   "metadata": {},
   "source": [
    "当前案例中，可以将猫猫类别作为阳性样本，也就是二分类中的1类，狗狗作为阴性数据，也就是0类样本\n",
    "对于刚才的案例而言，P = 30， N = 70， PP = 40, PN = 60"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20495056",
   "metadata": {},
   "source": [
    "进行二分类模型预测过程中，样本类别被模型正确识别的情况其实有两种，一种是阳性样本被正确识别，另一种是阴性样本被正确识别，据此我们可以有如下定义：\n",
    "- True positive（TP）：样本属于阳性（类别1）、并且被正确识别为阳性（类别1）的样本总数；TP发生时也被称为正确命中（hit）；\n",
    "- True negative（TN）：样本属于阴性（类别0）、并且被正确识别为阴性（类别0）的样本总数；TN发生时也被称为正确拒绝（correct rejection）；"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb8198f9",
   "metadata": {},
   "source": [
    "上述样本中，TP=25，TN = 55"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6efd3751",
   "metadata": {},
   "source": [
    " 当然，对于误分类的样本，其实也有两种情况，其一是阳性样本被误识别为阴性，其二是阴性样本被误识别为阳性，据此我们也有如下定义\n",
    "- False positive（FP）：样本属于阴性（类别0），但被错误判别为阳性（类别1）的样本总数；FP发生时也被称为发生I类了错误（Type I error），或者假警报（False alarm）、低估（underestimation）等；\n",
    "- False negative（FN）：样本属于阳性（类别1），但被错误判别为阴性（类别0）的样本总数；FN发生时也被称为发生了II类错误（Type II error），或者称为错过目标（miss）、高估（overestimation）等；\n",
    "\n",
    "上述样本中 FP = 15，FN = 5\n",
    "\n",
    "混淆矩阵也可以写成如下形式\n",
    "\n",
    "\n",
    "<img src=\"./images/09.jpg\" alt=\"1\" />\n",
    "\n",
    "其中准确率 \n",
    "\n",
    "$$\n",
    "Accuracy = \\frac{TP+TN}{TP+TN+FP+FN} = \\frac{25+55}{25+5+15+55} * 100\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd91b87d",
   "metadata": {},
   "source": [
    "但是，准确率指标并不总是能够评估一个模型的好坏，比如对于下面的情况，假如有一个数据集，含有98个狗狗，2个猫，而分类器model,是一个很差劲的分类器，它把数据集的所有样本都划分为狗狗，也就是不管输入什么样的样本，该模型都认为该样本是狗狗。那么这个表格会是什么样的了\n",
    "\n",
    "<img src=\"./images/10.jpg\" alt=\"1\" />\n",
    "\n",
    "则该模型的准确率为98%，因为它正确地识别出来了测试集中的98个狗狗，只是错误的把2个猫咪也当做狗狗，所以按照准确率的计算公式，该模型有高达98%的准确率。\n",
    "\n",
    "可是，这样的模型有意义吗？我们主要想识别出猫猫的类别，特意把猫猫作为1类，但是当前模型为了尽量追求准确率，完全牺牲了对猫猫识别的精度，这是一个极端的情况，却又是普遍的情况，准确率在一些场景并不适用，特别是对于这种样品数量偏差比较大的问题，准确率的“准确度”会极大的下降。所以，这时就需要引入其他评估指标评价模型的好坏了。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40ea0f06",
   "metadata": {},
   "source": [
    "- 召回率（Recall）\n",
    "召回率侧重于关注全部的1类样本中别准确识别出来的比例，其计算公式为\n",
    "$$\n",
    "Recall = \\frac{TP}{TP+FN}\n",
    "$$\n",
    "对于当前案例，我们的召回率是 25 / (25+5) = 0.833, 30条正例样本，其中25条被预测正确"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5cab0b00",
   "metadata": {},
   "source": [
    "根据召回率的计算公式我们可以试想，如果以召回率作为模型评估指标，则会使得模型非常重视是否把1全部识别了出来，甚至是牺牲掉一些0类样本判别的准确率来提升召回率，即哪怕是错判一些0样本为1类样本，也要将1类样本识别出来，这是一种“宁可错杀一千不可放过一个”的判别思路。因此，召回率其实是一种较为激进的识别1类样本的评估指标，在0类样本被误判代价较低、而1类样本被误判成本较高时可以考虑使用。“宁可错杀一千不可放过一个"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7bc7a57b",
   "metadata": {},
   "source": [
    "当然，对于极度不均衡样本，这种激进的判别指标也能够很好的判断模型有没有把1类样本成功的识别出来。例如总共100条数据，其中有99条样本标签为0、剩下一条样本标签为1，假设模型总共有A、B、C三个模型，A模型判别所有样本都为0类，B模型判别50条样本为1类50条样本为0类，并且成功识别唯一的一个1类样本，C模型判别20条样本为1类、80条样本为0类，同样成功识别了唯一的一个1类样本，则各模型的准确率和召回率如下："
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47adb46b",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-23T10:38:35.649506Z",
     "start_time": "2023-05-23T10:38:35.644436Z"
    }
   },
   "source": [
    "|Model|ACC|Recall|\n",
    "|:--:|:--:|:--:|\n",
    "|A|99%|0|\n",
    "|B|51%|100%|\n",
    "|C|81%|100%|"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2aefb4ad",
   "metadata": {},
   "source": [
    "  不难发现，在偏态数据中，相比准确率，召回率对于1类样本能否被正确识别的敏感度要远高于准确率，但对于是否牺牲了0类别的准确率却无法直接体现。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60697f7c",
   "metadata": {},
   "source": [
    "- 精确率（Precision）"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06cbf636",
   "metadata": {},
   "source": [
    "精确率的定义是：对于给定测试集的某一个类别，分类模型预测正确的比例，或者说：分类模型预测的正样本中有多少是真正的正样本，其计算公式是\n",
    "\n",
    "$$\n",
    "Precision = \\frac{TP}{TP+FP}\n",
    "$$\n",
    "\n",
    "当前案例中，Precision = 25 / 25 + 15 = 0.625\n",
    "\n",
    "精确度，衡量对1类样本的识别，能否成功（准确识别出1）的概率，也正是由于这种力求每次出手都尽可能成功的策略，使得当我们在以精确度作为模型判别指标时，模型整体对1的判别会趋于保守，只对那些大概率确定为1的样本进行1类的判别，从而会一定程度牺牲1类样本的准确率，在每次判别成本较高、而识别1样本获益有限的情况可以考虑使用精确度"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f7aa81c",
   "metadata": {},
   "source": [
    " 关于召回率和精确度，也可以通过如下形式进行更加形象的可视化展示\n",
    " \n",
    " <img src=\"./images/11.jpg\" alt=\"1\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1e3b5d5",
   "metadata": {},
   "source": [
    "-  F1值（F1-Measure)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a08ccc2b",
   "metadata": {},
   "source": [
    "在理想情况下，我们希望模型的精确率越高越好，同时召回率也越高越高，但是，现实情况往往事与愿违，在现实情况下，精确率和召回率像是坐在跷跷板上一样，往往出现一个值升高，另一个值降低，那么，有没有一个指标来综合考虑精确率和召回率了，再大多数情况下，其实我们是希望获得一个更加“均衡”的模型判别指标，即我们既不希望模型太过于激进、也不希望模型太过于保守，并且对于偏态样本，既可以较好的衡量1类样本是否被识别，同时也能够兼顾考虑到0类样本的准确率牺牲程度，此时，我们可以考虑使用二者的调和平均数（harmonic mean）作为模型评估指标，这个指标就是F值。F值的计算公式为"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60ca4a7c",
   "metadata": {},
   "source": [
    "$$\n",
    "F1-Score = \\frac{2}{\\frac{1}{Recall}+\\frac{1}{Precision}}=\\frac{2 \\cdot Recall \\cdot Precision}{Recall+Precision}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9123110",
   "metadata": {},
   "source": [
    "F1-Score指标能够一定程度上综合Recall和Precision的结果，综合判断模型整体分类性能。当然，除了F1-Score以外我们还可以取Recall和Precision的均值(balanced accuracy，简称BA)来作为模型评估指标"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e80a9b8f",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-23T11:34:34.946142Z",
     "start_time": "2023-05-23T11:34:34.934949Z"
    }
   },
   "source": [
    "### sklearn 中的指标计算"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "08f690b8",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T03:29:41.721341Z",
     "start_time": "2023-05-26T03:29:40.145780Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import recall_score, precision_score, f1_score"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c08958d",
   "metadata": {},
   "source": [
    "使用方法都是一致的，直接调用函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "e745cffb",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T03:30:26.743038Z",
     "start_time": "2023-05-26T03:30:26.721741Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "召回率:0.5\n",
      "精确率:0.6666666666666666\n",
      "f1-score:0.5714285714285715\n"
     ]
    }
   ],
   "source": [
    "y_true = [0, 1, 1, 0, 1, 1]\n",
    "y_pred = [0, 0, 1, 1, 1, 0]\n",
    "print(f\"召回率:{recall_score(y_true, y_pred)}\")  \n",
    "print(f\"精确率:{precision_score(y_true, y_pred)}\")  \n",
    "print(f\"f1-score:{f1_score(y_true, y_pred)}\")  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "bc2a5d6b",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-24T03:30:36.161829Z",
     "start_time": "2023-05-24T03:30:36.149068Z"
    }
   },
   "outputs": [],
   "source": [
    "#请在此编写代码\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c11bec87",
   "metadata": {},
   "source": [
    "### 总结\n",
    "\n",
    "- 在类别划分上，仍然需要强调的是，我们需要根据实际业务情况，将重点识别的样本类划为类别1，其他样本划为类别0\n",
    "- 如果0、1两类在业务判断上并没有任何重要性方面的差异，那么我们可以将样本更少的哪一类划为1类\n",
    "- 在评估指标选取上，同样需要根据业务情况判断，如果只需要考虑1类别的识别率，则可考虑使用Recall作为模型评估指标，若只需考虑对1样本判别结果中的准确率，则可考虑使用Precision作为评估指标。但一般来说这两种情况其实都不多，更普遍的情况是，需要重点识别1类但也要兼顾0类的准确率，此时我们可以使用F1-Score指标。F1-Score其实也是分类模型中最为通用和常见的分类指标"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f464b78",
   "metadata": {},
   "source": [
    "## 归一化"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46c61415",
   "metadata": {},
   "source": [
    "### 特征缩放"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44943257",
   "metadata": {},
   "source": [
    " 因为对于大多数的机器学习算法和优化算法来说，将特征值缩放到相同区间可以使得获取性能更好的模型。就梯度下降算法而言，例如有两个不同的特征，第一个特征的取值范围为1——10，第二个特征的取值范围为1——10000。在梯度下降算法中，代价函数为最小平方误差函数，所以在使用梯度下降算法的时候，算法会明显的偏向于第二个特征，因为它的取值范围更大。在比如，k近邻算法，它使用的是欧式距离，也会导致其偏向于第二个特征。对于决策树和随机森林以及XGboost算法而言，特征缩放对于它们没有什么影响。\n",
    "\n",
    "  常用的特征缩放算法有两种，归一化(normalization)和标准化(standardization)。归一化算法是通过特征的最大最小值将特征缩放到[0,1]区间范围"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1a6f8cf",
   "metadata": {},
   "source": [
    "###  归一化(Normalization)\n",
    "  归一化是利用特征的最大最小值，为了方便数据处理，将特征的值缩放到[0,1]区间，对于每一列的特征使用min-max函数进行缩放，可以使处理过程更加便捷、快速，计算公式如下：\n",
    "    \n",
    "   $$\\frac{X_i-X_{min}}{X_{max}-X_{min}}$$\n",
    "   \n",
    "  其中$X_{max}$为最大值，$X_{min}$为最小值"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e7852ba",
   "metadata": {},
   "source": [
    "### 特征归一化的优势\n",
    "\n",
    "- 提升收敛速度 ：对于线性model来说，数据归一化后，最优解的寻优过程明显会变得平缓，更容易正确的收敛到最优解。\n",
    "  \n",
    "  数据没有进行归一化的表达式，可以为：\n",
    "  \n",
    "  $$J=(3\\theta_1+600\\theta_2-y_{correct})^2$$\n",
    "  \n",
    "  造成图像的等高线为类似椭圆形状，最优解的寻优过程图像如下：\n",
    "   <img src=\"./images/12.jpg\" alt=\"1\" />\n",
    "   \n",
    "   两个特征区别相差特别大。所形成的等高线比较尖锐。当时用梯度下降法时，很可能要垂直等高线走，需要很多次迭代才能收敛。\n",
    "   而数据归一化之后，损失函数的表达式可以表示为：\n",
    "  \n",
    "   $$ J=(0.5\\theta_1+0.55\\theta_2-y_{correct})^2$$\n",
    "   \n",
    "   其中变量的前面系数几乎一样，则图像的等高线为类似圆形形状，最优解的寻优过程图像如下：\n",
    "   \n",
    "   <img src=\"./images/13.jpg\" alt=\"1\" />\n",
    "   \n",
    "    对两个原始特征进行了归一化处理，其对应的等高线相对来说比较圆，在梯度下降时，可以较快的收敛。\n",
    "    \n",
    " - 提升模型精度：如果我们选用的距离度量为欧式距离，如果数据预先没有经过归一化，那么那些绝对值大的features在欧式距离计算的时候起了决定性作用。 从经验上说，归一化是让不同维度之间的特征在数值上有一定比较性，可以大大提高分类器的准确性。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "728b63eb",
   "metadata": {},
   "source": [
    "### 特征归一化方法MinMaxScaler\n",
    "\n",
    "在sklearn中，sklearn.preprocessing.MinMaxScaler是一种用于特征归一化的方法。\n",
    "\n",
    "  示例1如下：\n",
    "```\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "x=[[10001,2],[16020,4],[12008,6],[13131,8]]\n",
    "min_max_scaler = MinMaxScaler()\n",
    "X_train_minmax = min_max_scaler.fit_transform(x)#归一化后的结果\n",
    "X_train_minmax\n",
    "```\n",
    " 它默认将每种特征的值都归一化到[0，1]之间"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "e217f911",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T03:59:22.991202Z",
     "start_time": "2023-05-26T03:59:22.982323Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler\n",
    "x=[[10001,2],[16020,4],[12008,6],[13131,8],[13222,12]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "ee0f33b8",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T03:59:24.213666Z",
     "start_time": "2023-05-26T03:59:24.200724Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-4 {color: black;background-color: white;}#sk-container-id-4 pre{padding: 0;}#sk-container-id-4 div.sk-toggleable {background-color: white;}#sk-container-id-4 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-4 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-4 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-4 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-4 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-4 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-4 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-4 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-4 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-4 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-4 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-4 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-4 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-4 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-4 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-4 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-4 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-4 div.sk-item {position: relative;z-index: 1;}#sk-container-id-4 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-4 div.sk-item::before, #sk-container-id-4 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-4 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-4 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-4 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-4 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-4 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-4 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-4 div.sk-label-container {text-align: center;}#sk-container-id-4 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-4 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-4\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>MinMaxScaler()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-4\" type=\"checkbox\" checked><label for=\"sk-estimator-id-4\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">MinMaxScaler</label><div class=\"sk-toggleable__content\"><pre>MinMaxScaler()</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "MinMaxScaler()"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mm = MinMaxScaler()\n",
    "# mm.fit_transform(x) # fit _ transform\n",
    "# mm.fit(x)\n",
    "# mm.transform(x)\n",
    "xtrain = x[:3]\n",
    "xtest = x[3:]\n",
    "mm.fit(xtrain)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "f7f15335",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T03:59:24.855200Z",
     "start_time": "2023-05-26T03:59:24.841634Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.        , 0.        ],\n",
       "       [1.        , 0.5       ],\n",
       "       [0.33344409, 1.        ]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mm.transform(xtrain)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "39436e8b",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T03:59:25.808403Z",
     "start_time": "2023-05-26T03:59:25.798874Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[13131, 8], [13222, 12]]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xtest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "7f409526",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T03:59:30.419029Z",
     "start_time": "2023-05-26T03:59:30.407425Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.52001994, 1.5       ],\n",
       "       [0.53513873, 2.5       ]])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mm.transform(xtest)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ffe9e9",
   "metadata": {},
   "source": [
    " 归一化后的数值大小范围是可调的（根据MinMaxScaler的参数feature_range调整）\n",
    " <br>\n",
    " 示例2：能将特征归一化到[-1,1]之间。\n",
    "```\n",
    "min_max_scaler = MinMaxScaler(feature_range=(-1,1))\n",
    "X_train_minmax = min_max_scaler.fit_transform(x)#归一化后的结果\n",
    "X_train_minmax\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "225f0e47",
   "metadata": {},
   "source": [
    "####  MinMaxScaler的实现\n",
    "  MinMaxScaler的实现公式如下:\n",
    "```\n",
    "X_std = (X - X.min(axis=0)) / (X.max(axis=0) - X.min(axis=0))\n",
    "X_scaled = X_std * (max - min) + min\n",
    "```\n",
    "  这是向量化的表达方式，说明X是矩阵，其中\n",
    "* X_std：将X归一化到[0，1]之间\n",
    "* X.min(axis=0)表示列最小值\n",
    "* max，min表示MinMaxScaler的参数feature_range参数。即最终结果的大小范围\n",
    "  以下例说明计算过程（max=1，min=0）\n",
    "\n",
    " <img src=\"./images/14.jpg\" alt=\"1\" />\n",
    "\n",
    "  归一化的过程如下，假设归一化后的矩阵为S\n",
    "```\n",
    "S11=(10001-10001)/(16020-10001)=0\n",
    "S21=(16020-10001)/(16020-10001)=1\n",
    "S31=(12008-10001)/(16020-10001)=0.333444\n",
    "S41=(13131-10001)/(16020-10001)=0.52002\n",
    "S12=(2-2)/(8-2)=0\n",
    "S22=(4-2)/(8-2)=0.33\n",
    "S32=(6-2)/(8-2)=0.6667\n",
    "S42=(8-2)/(8-2)=1\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4048da1e",
   "metadata": {},
   "source": [
    "数据的标准化"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "776457c8",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T03:35:53.634739Z",
     "start_time": "2023-05-26T03:35:53.624770Z"
    }
   },
   "source": [
    "&emsp;&emsp;和0-1标准化不同，Z-score标准化利用原始数据的均值（mean）和标准差（standard deviation）进行数据的标准化。同样是逐列进行操作，每一条数据都减去当前列的均值再除以当前列的标准差，在这种标准化操作下，如果原数据服从正态分布，处理之后的数据服从标准正态分布。Z-Score标准化计算公式如下：\n",
    "\n",
    "$${x}_{normalization}=\\frac{x-\\mu }{\\sigma }$$\n",
    "\n",
    "其中 𝜇\n",
    " 代表均值， 𝜎\n",
    " 代表标准差。当然，我们也可通过如下方式对张量进行Z-Score标准化处理。\n",
    " \n",
    "```python\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "x=[[10001,2],[16020,4],[12008,6],[13131,8]]\n",
    "ss = StandardScaler()\n",
    "X_train = ss.fit_transform(x)\n",
    "X_train\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "862f1b45",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T05:51:51.523894Z",
     "start_time": "2023-05-26T05:51:51.508307Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-1.2817325 , -1.34164079],\n",
       "       [ 1.48440157, -0.4472136 ],\n",
       "       [-0.35938143,  0.4472136 ],\n",
       "       [ 0.15671236,  1.34164079]])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "x=[[10001,2],[16020,4],[12008,6],[13131,8]]\n",
    "ss = StandardScaler()\n",
    "X_train = ss.fit_transform(x)\n",
    "X_train"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29939c90",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T03:39:33.302635Z",
     "start_time": "2023-05-26T03:39:33.292453Z"
    }
   },
   "source": [
    "和0-1标准化不同，Z-Score标准化并不会将数据放缩在0-1之间，而是均匀地分布在0的两侧"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "daa0e071",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-24T03:40:41.141668Z",
     "start_time": "2023-05-24T03:40:41.130573Z"
    }
   },
   "source": [
    "## 特征编码"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "baa44e8a",
   "metadata": {},
   "source": [
    "我们拿到的数据通常比较脏乱，特征变量除了数值外可能还会包括带有各种非数字特殊符号等特征值，比如中文。但一般的机器学习模型一般都是处理数值型的特征值，因此需要将一些非数值的特殊特征值转为为数值，因为只有数字类型才能进行计算。因此，对于各种特殊的特征值，我们都需要对其进行相应的编码，也是量化的过程，这就要用到特征编码"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9a223b3",
   "metadata": {},
   "source": [
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5527fbd5",
   "metadata": {},
   "source": [
    "### 编码方法"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9835a5d5",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-24T05:49:19.035935Z",
     "start_time": "2023-05-24T05:49:19.026274Z"
    }
   },
   "source": [
    "- LabelEncoder ：适合处理字符型数据或label类，一般先用此方法将字符型数据转换为数值型，然后再用以下两种方法编码；\n",
    "- get_dummies ：pandas 方法，处理DataFrame 数据更便捷\n",
    "- OneHotEncoder ：更普遍的编码方法\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af46845a",
   "metadata": {},
   "source": [
    "### LabelEncoder\n",
    "\n",
    "label-encoding就是用标签进行编码的意思，即我们给特征变量自定义数字标签，量化特征。\n",
    "\n",
    "  将离散的数值或字符串，转化为连续的数值型数据。n个类别就用0到n-1个数表示。没有扩维，多用于标签列的编码（如果用于特征的编码，那编码后还要用get_dummies或OneHotEncoder进行再编码，才能实现扩维）。\n",
    "\n",
    "  实例如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "02f9fab9",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T06:01:00.804595Z",
     "start_time": "2023-05-26T06:00:59.933809Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>大一</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>大二</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>大三</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>大四</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Class\n",
       "0    大一\n",
       "1    大二\n",
       "2    大三\n",
       "3    大四"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "Class=['大一','大二','大三','大四']\n",
    "df = pd.DataFrame({'Class':Class})\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d52afffc",
   "metadata": {},
   "source": [
    "Label encoding就是对特征值进行自定义式的标签编码。比如将大一变为1，大二变为2，大三为3，大四为4。那么经过编码后，特征值将变为以下的样子"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "6735aaec",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T06:02:03.943282Z",
     "start_time": "2023-05-26T06:02:03.923508Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Class\n",
       "0      1\n",
       "1      2\n",
       "2      3\n",
       "3      4"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classMap = {'大一':1,'大二':2,'大三':3,'大四':4}\n",
    "df['Class'] = df['Class'].map(classMap)\n",
    "df\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "954aa325",
   "metadata": {},
   "source": [
    "上面就将Class特征进行相应的编码。其实，Label encoding并没有任何限制，你也可以将Class定义为10,20,30,40，只不过1,2,3,4看起来比较方便。因此总结概括，Label encoding就是将原始特征值编码为自定义的数字标签完成量化编码过程"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd878329",
   "metadata": {},
   "source": [
    "### get_dummies"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "095e0a6c",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-24T05:51:13.089134Z",
     "start_time": "2023-05-24T05:51:13.072221Z"
    }
   },
   "source": [
    " pandas编码工具，直接将数据扩维\n",
    "\n",
    "  \n",
    "  pandas 中函数的格式如下：\n",
    "```\n",
    "pandas.get_dummies(data, prefix=None, prefix_sep='_', dummy_na=False, columns=None, sparse=False, drop_first=False, dtype=None)\n",
    "```\n",
    "\n",
    "  实例一如下：\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "5a427da1",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T06:03:08.619963Z",
     "start_time": "2023-05-26T06:03:08.600374Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>color</th>\n",
       "      <th>size</th>\n",
       "      <th>length</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>green</td>\n",
       "      <td>M</td>\n",
       "      <td>10.1</td>\n",
       "      <td>label1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>red</td>\n",
       "      <td>L</td>\n",
       "      <td>13.5</td>\n",
       "      <td>label2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>blue</td>\n",
       "      <td>XL</td>\n",
       "      <td>15.3</td>\n",
       "      <td>label2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   color size  length   label\n",
       "0  green    M    10.1  label1\n",
       "1    red    L    13.5  label2\n",
       "2   blue   XL    15.3  label2"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd \n",
    "df = pd.DataFrame([  \n",
    "            ['green', 'M', 10.1, 'label1'],   \n",
    "            ['red', 'L', 13.5, 'label2'],   \n",
    "            ['blue', 'XL', 15.3, 'label2']])  \n",
    "# color、label不具备大小含义，size具有大小意义\n",
    "df.columns = ['color', 'size', 'length', 'label'] \n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "1c02a25e",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T06:04:14.899381Z",
     "start_time": "2023-05-26T06:04:14.883294Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>length</th>\n",
       "      <th>color_blue</th>\n",
       "      <th>color_green</th>\n",
       "      <th>color_red</th>\n",
       "      <th>size_L</th>\n",
       "      <th>size_M</th>\n",
       "      <th>size_XL</th>\n",
       "      <th>label_label1</th>\n",
       "      <th>label_label2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10.1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>13.5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15.3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   length  color_blue  color_green  color_red  size_L  size_M  size_XL  \\\n",
       "0    10.1           0            1          0       0       1        0   \n",
       "1    13.5           0            0          1       1       0        0   \n",
       "2    15.3           1            0          0       0       0        1   \n",
       "\n",
       "   label_label1  label_label2  \n",
       "0             1             0  \n",
       "1             0             1  \n",
       "2             0             1  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    " ##如果不指定data列的话，默认是所有的分类变量进行one_hot处理\n",
    "df_1=pd.get_dummies(df)\n",
    "df_1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9ff023d",
   "metadata": {},
   "source": [
    "  prefix参数可以给哑变量的名字加上一个前缀"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "a16aca97",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T06:05:52.622106Z",
     "start_time": "2023-05-26T06:05:52.605093Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>color1_blue</th>\n",
       "      <th>color1_green</th>\n",
       "      <th>color1_red</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   color1_blue  color1_green  color1_red\n",
       "0            0             1           0\n",
       "1            0             0           1\n",
       "2            1             0           0"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_2 =pd.get_dummies(df[\"color\"],prefix=\"color1\")\n",
    "df_2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8bf5f75e",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-24T05:52:51.238854Z",
     "start_time": "2023-05-24T05:52:51.214779Z"
    }
   },
   "source": [
    "同时在pandas可以指定 columns参数，pd.get_dummies(df,columns=[\"length\",\"size\"])指定被编码的列,返回被编码的列和不被编码的列"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "5f3fa945",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T06:06:52.433740Z",
     "start_time": "2023-05-26T06:06:52.409680Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>color</th>\n",
       "      <th>label</th>\n",
       "      <th>length_10.1</th>\n",
       "      <th>length_13.5</th>\n",
       "      <th>length_15.3</th>\n",
       "      <th>size_L</th>\n",
       "      <th>size_M</th>\n",
       "      <th>size_XL</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>green</td>\n",
       "      <td>label1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>red</td>\n",
       "      <td>label2</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>blue</td>\n",
       "      <td>label2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   color   label  length_10.1  length_13.5  length_15.3  size_L  size_M  \\\n",
       "0  green  label1            1            0            0       0       1   \n",
       "1    red  label2            0            1            0       1       0   \n",
       "2   blue  label2            0            0            1       0       0   \n",
       "\n",
       "   size_XL  \n",
       "0        0  \n",
       "1        0  \n",
       "2        1  "
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_4 =pd.get_dummies(df,columns=[\"length\",\"size\"])\n",
    "df_4"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4983d681",
   "metadata": {},
   "source": [
    "### OneHotEncoder"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74ad5724",
   "metadata": {},
   "source": [
    "当然，除了自然顺序编码外，常见的对离散变量的编码方式还有独热编码，独热编码的过程如下"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f32d3e2",
   "metadata": {},
   "source": [
    " <img src=\"./images/15.jpg\" alt=\"1\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd707b3f",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-24T05:55:32.781879Z",
     "start_time": "2023-05-24T05:55:32.769028Z"
    }
   },
   "source": [
    "不难发现，独热编码过程其实和我们此前介绍的哑变量创建过程一致（至少在sklearn中并无差别）。对于独热编码的过程，我们可以通过pd.get_dummies函数实现，也可以通过sklearn中OneHotEncoder评估器（转化器）来实现。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "6aff9df8",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T06:13:21.943033Z",
     "start_time": "2023-05-26T06:13:21.935002Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import OneHotEncoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "daa2f807",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "a54482e1",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T06:13:26.836456Z",
     "start_time": "2023-05-26T06:13:26.822985Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([['F'],\n",
       "       ['M'],\n",
       "       ['M'],\n",
       "       ['F']], dtype='<U1')"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X1 = np.array([['F'], ['M'], ['M'], ['F']])\n",
    "X1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "3af66f2b",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T06:15:50.113726Z",
     "start_time": "2023-05-26T06:15:50.107862Z"
    }
   },
   "outputs": [],
   "source": [
    "one = OneHotEncoder()\n",
    "# sparse=False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "9f1e566a",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T06:15:50.963006Z",
     "start_time": "2023-05-26T06:15:50.948433Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-6 {color: black;background-color: white;}#sk-container-id-6 pre{padding: 0;}#sk-container-id-6 div.sk-toggleable {background-color: white;}#sk-container-id-6 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-6 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-6 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-6 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-6 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-6 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-6 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-6 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-6 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-6 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-6 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-6 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-6 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-6 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-6 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-6 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-6 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-6 div.sk-item {position: relative;z-index: 1;}#sk-container-id-6 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-6 div.sk-item::before, #sk-container-id-6 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-6 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-6 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-6 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-6 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-6 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-6 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-6 div.sk-label-container {text-align: center;}#sk-container-id-6 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-6 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-6\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>OneHotEncoder()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-6\" type=\"checkbox\" checked><label for=\"sk-estimator-id-6\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">OneHotEncoder</label><div class=\"sk-toggleable__content\"><pre>OneHotEncoder()</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "OneHotEncoder()"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "one.fit(X1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "1b4ab1ba",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T06:16:24.156254Z",
     "start_time": "2023-05-26T06:16:24.141857Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1., 0.],\n",
       "       [0., 1.]])"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "one.transform([[\"F\"],[\"M\"]]).toarray()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3cc34bd9",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-24T06:04:41.367020Z",
     "start_time": "2023-05-24T06:04:41.361727Z"
    }
   },
   "source": [
    "同样，训练完成后的转化器会记录转化规则："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "1cae3fd3",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T06:16:58.263014Z",
     "start_time": "2023-05-26T06:16:58.252405Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array(['F', 'M'], dtype='<U1')]"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "one.categories_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7f64e2a",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-24T06:06:09.747218Z",
     "start_time": "2023-05-24T06:06:09.738738Z"
    }
   },
   "source": [
    "- drop='if_binary'过程"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b52757d4",
   "metadata": {},
   "source": [
    " 对于独热编码的使用，有一点是额外需要注意的，那就是对于二分类离散变量来说，独热编码往往是没有实际作用的。例如对于上述极简数据集而言，Gender的取值是能是M或者F，独热编码转化后，某行Gender_F取值为1、则Gender_M取值必然为0，反之亦然。因此很多时候我们在进行独热编码转化的时候会考虑只对多分类离散变量进行转化，而保留二分类离散变量的原始取值。此时就需要将OneHotEncoder中drop参数调整为'if_binary'，以表示跳过二分类离散变量列"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "30a4cd31",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T06:18:58.602127Z",
     "start_time": "2023-05-26T06:18:58.583769Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Gender</th>\n",
       "      <th>Income</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>F</td>\n",
       "      <td>High</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>M</td>\n",
       "      <td>Medium</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>M</td>\n",
       "      <td>High</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>F</td>\n",
       "      <td>Low</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Gender  Income\n",
       "0      F    High\n",
       "1      M  Medium\n",
       "2      M    High\n",
       "3      F     Low"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X3 = pd.DataFrame({'Gender': ['F', 'M', 'M', 'F'], 'Income': ['High', 'Medium', 'High', 'Low']})\n",
    "X3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "b7f2e5bf",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T06:18:59.661587Z",
     "start_time": "2023-05-26T06:18:59.644847Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0., 1., 0., 0.],\n",
       "       [1., 0., 0., 1.],\n",
       "       [1., 1., 0., 0.],\n",
       "       [0., 0., 1., 0.]])"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "drop_enc = OneHotEncoder(drop='if_binary')\n",
    "drop_enc.fit_transform(X3).toarray()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "79662035",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T06:19:03.895296Z",
     "start_time": "2023-05-26T06:19:03.886450Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array(['F', 'M'], dtype=object),\n",
       " array(['High', 'Low', 'Medium'], dtype=object)]"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "drop_enc.categories_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f71d8760",
   "metadata": {},
   "source": [
    "#### 总结\n",
    "get_dummies与OneHotEncoder对比\n",
    "1. 本身就是 pandas 的模块，所以对 DataFrame 类型兼容很好.\n",
    "2. 无论你的列是字符型还是数字型都可以进行二值编码.\n",
    "3. 能根据用户指定，自动生成二值编码后的变量名.\n",
    "\n",
    "不同\n",
    "\n",
    "1. 不是 sklearn 里的transformer类型，所以得到的结果得手动输入到 sklearn 里的相应模块\n",
    "2. 无法像 sklearn 的transformer一样可以输入到pipeline中 进行流程化地机器学习过程\n",
    "3. get_dummies 不像 sklearn 的 transformer一样，有 transform方法，所以一旦测试集中出现了训练集未曾出现过的特征取值，简单地对测试集、训练集都用 get_dummies 方法将导致数据错误。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ebd49ba",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-24T06:30:09.540462Z",
     "start_time": "2023-05-24T06:30:09.534155Z"
    }
   },
   "source": [
    "## sklearn中逻辑回归的参数解释"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d2ecc8a",
   "metadata": {},
   "source": [
    "接下来我们对逻辑回归评估器中的参数进行详细解释："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "33ca07b2",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T08:36:24.363756Z",
     "start_time": "2023-05-26T08:36:24.359301Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1f670f2",
   "metadata": {},
   "source": [
    "|参数|解释|\n",
    "|:--:|:--:|    \n",
    "|penalty|正则化项|   \n",
    "|dual|是否求解对偶问题*|\n",
    "|tol|迭代停止条件：两轮迭代损失值差值小于tol时，停止迭代|\n",
    "|C|经验风险和结构风险在损失函数中的权重|\n",
    "|fit_intercept|线性方程中是否包含截距项|\n",
    "|intercept_scaling|相当于此前讨论的特征最后一列全为1的列，当使用liblinear求解参数时用于捕获截距|\n",
    "|class_weight|各类样本权重*|\n",
    "|random_state|随机数种子|\n",
    "|solver|损失函数求解方法*|\n",
    "|max_iter|求解参数时最大迭代次数，迭代过程满足max_iter或tol其一即停止迭代|\n",
    "|multi_class|多分类问题时求解方法*|\n",
    "|verbose|是否输出任务进程|\n",
    "|warm_start|是否使用上次训练结果作为本次运行初始参数|\n",
    "|l1_ratio|当采用弹性网正则化时，$l1$正则项权重，就是损失函数中的$\\rho$|"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7eaa08fb",
   "metadata": {},
   "source": [
    "- C  惩罚系数 \n",
    "- penalty 正则化项"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29343175",
   "metadata": {},
   "source": [
    " <img src=\"./images/16.jpg\" alt=\"1\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ff5ac5e",
   "metadata": {},
   "source": [
    " <img src=\"./images/22.jpg\" alt=\"1\" />\n",
    " \n",
    " <img src=\"./images/23.jpg\" alt=\"1\" />\n",
    "  \n",
    " <img src=\"./images/24.jpg\" alt=\"1\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "144ea93d",
   "metadata": {},
   "source": [
    "&emsp;&emsp;相比原始损失函数，正则化后的损失函数有两处发生了变化，其一是在原损失函数基础上乘以了系数C，其二则是加入了正则化项。其中系数C也是超参数，需要人工输入，用于调整经验风险部分和结构风险部分的权重，C越大，经验风险部分权重越大，反之结构风险部分权重越大。此外，在$l2$正则化时，采用的$\\frac{w^Tw}{2}$表达式，其实相当于是各参数的平方和除以2，在求最小值时本质上和w的2-范数起到的作用相同，省去开平方是为了简化运算，而除以2则是为了方便后续求导运算，和2次方结果相消。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "109a4bad",
   "metadata": {},
   "source": [
    "- solver：损失函数求解方法\n",
    "\n",
    "其实除了最小二乘法和梯度下降以外，还有非常多的关于损失函数的求解方法，而选择损失函数的参数，就是solver参数。\n",
    "\n",
    " <img src=\"./images/17.jpg\" alt=\"1\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "924f94c7",
   "metadata": {},
   "source": [
    "&emsp;&emsp;逻辑回归可选的优化方法包括：        \n",
    "&emsp;&emsp;liblinear，这是一种坐标轴下降法，并且该软件包中大多数算法都有C++编写，运行速度很快，支持OVR+L1或OVR+L2；      \n",
    "&emsp;&emsp;lbfgs，全称是L-BFGS，牛顿法的一种改进算法（一种拟牛顿法），适用于小型数据集，并且支持MVM+L2、OVR+L2以及不带惩罚项的情况；      \n",
    "&emsp;&emsp;newton-cg，同样也是一种拟牛顿法，和lbfgs适用情况相同；      \n",
    "&emsp;&emsp;sag，随机平均梯度下降，随机梯度下降的改进版，类似动量法，会在下一轮随机梯度下降开始之前保留一些上一轮的梯度，从而为整个迭代过程增加惯性，除了不支持L1正则化的损失函数求解以外（包括弹性网正则化）其他所有损失函数的求解；      \n",
    "&emsp;&emsp;saga，sag的改进版，修改了梯度惯性的计算方法，使得其支持所有情况下逻辑回归的损失函数求解；"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90b4465d",
   "metadata": {},
   "source": [
    "### 手动参数调整"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02338a1c",
   "metadata": {},
   "source": [
    "导入我们的工具包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "00ac401e",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T08:52:54.774608Z",
     "start_time": "2023-05-26T08:52:54.081178Z"
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ac764b4",
   "metadata": {},
   "source": [
    "数据读取"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "a133327b",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T08:52:59.779307Z",
     "start_time": "2023-05-26T08:52:59.740874Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Time</th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "      <th>V5</th>\n",
       "      <th>V6</th>\n",
       "      <th>V7</th>\n",
       "      <th>V8</th>\n",
       "      <th>V9</th>\n",
       "      <th>...</th>\n",
       "      <th>V21</th>\n",
       "      <th>V22</th>\n",
       "      <th>V23</th>\n",
       "      <th>V24</th>\n",
       "      <th>V25</th>\n",
       "      <th>V26</th>\n",
       "      <th>V27</th>\n",
       "      <th>V28</th>\n",
       "      <th>Amount</th>\n",
       "      <th>Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>406.0</td>\n",
       "      <td>-2.312227</td>\n",
       "      <td>1.951992</td>\n",
       "      <td>-1.609851</td>\n",
       "      <td>3.997906</td>\n",
       "      <td>-0.522188</td>\n",
       "      <td>-1.426545</td>\n",
       "      <td>-2.537387</td>\n",
       "      <td>1.391657</td>\n",
       "      <td>-2.770089</td>\n",
       "      <td>...</td>\n",
       "      <td>0.517232</td>\n",
       "      <td>-0.035049</td>\n",
       "      <td>-0.465211</td>\n",
       "      <td>0.320198</td>\n",
       "      <td>0.044519</td>\n",
       "      <td>0.177840</td>\n",
       "      <td>0.261145</td>\n",
       "      <td>-0.143276</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>472.0</td>\n",
       "      <td>-3.043541</td>\n",
       "      <td>-3.157307</td>\n",
       "      <td>1.088463</td>\n",
       "      <td>2.288644</td>\n",
       "      <td>1.359805</td>\n",
       "      <td>-1.064823</td>\n",
       "      <td>0.325574</td>\n",
       "      <td>-0.067794</td>\n",
       "      <td>-0.270953</td>\n",
       "      <td>...</td>\n",
       "      <td>0.661696</td>\n",
       "      <td>0.435477</td>\n",
       "      <td>1.375966</td>\n",
       "      <td>-0.293803</td>\n",
       "      <td>0.279798</td>\n",
       "      <td>-0.145362</td>\n",
       "      <td>-0.252773</td>\n",
       "      <td>0.035764</td>\n",
       "      <td>529.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4462.0</td>\n",
       "      <td>-2.303350</td>\n",
       "      <td>1.759247</td>\n",
       "      <td>-0.359745</td>\n",
       "      <td>2.330243</td>\n",
       "      <td>-0.821628</td>\n",
       "      <td>-0.075788</td>\n",
       "      <td>0.562320</td>\n",
       "      <td>-0.399147</td>\n",
       "      <td>-0.238253</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.294166</td>\n",
       "      <td>-0.932391</td>\n",
       "      <td>0.172726</td>\n",
       "      <td>-0.087330</td>\n",
       "      <td>-0.156114</td>\n",
       "      <td>-0.542628</td>\n",
       "      <td>0.039566</td>\n",
       "      <td>-0.153029</td>\n",
       "      <td>239.93</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>6986.0</td>\n",
       "      <td>-4.397974</td>\n",
       "      <td>1.358367</td>\n",
       "      <td>-2.592844</td>\n",
       "      <td>2.679787</td>\n",
       "      <td>-1.128131</td>\n",
       "      <td>-1.706536</td>\n",
       "      <td>-3.496197</td>\n",
       "      <td>-0.248778</td>\n",
       "      <td>-0.247768</td>\n",
       "      <td>...</td>\n",
       "      <td>0.573574</td>\n",
       "      <td>0.176968</td>\n",
       "      <td>-0.436207</td>\n",
       "      <td>-0.053502</td>\n",
       "      <td>0.252405</td>\n",
       "      <td>-0.657488</td>\n",
       "      <td>-0.827136</td>\n",
       "      <td>0.849573</td>\n",
       "      <td>59.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7519.0</td>\n",
       "      <td>1.234235</td>\n",
       "      <td>3.019740</td>\n",
       "      <td>-4.304597</td>\n",
       "      <td>4.732795</td>\n",
       "      <td>3.624201</td>\n",
       "      <td>-1.357746</td>\n",
       "      <td>1.713445</td>\n",
       "      <td>-0.496358</td>\n",
       "      <td>-1.282858</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.379068</td>\n",
       "      <td>-0.704181</td>\n",
       "      <td>-0.656805</td>\n",
       "      <td>-1.632653</td>\n",
       "      <td>1.488901</td>\n",
       "      <td>0.566797</td>\n",
       "      <td>-0.010016</td>\n",
       "      <td>0.146793</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 31 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Time        V1        V2        V3        V4        V5        V6  \\\n",
       "0   406.0 -2.312227  1.951992 -1.609851  3.997906 -0.522188 -1.426545   \n",
       "1   472.0 -3.043541 -3.157307  1.088463  2.288644  1.359805 -1.064823   \n",
       "2  4462.0 -2.303350  1.759247 -0.359745  2.330243 -0.821628 -0.075788   \n",
       "3  6986.0 -4.397974  1.358367 -2.592844  2.679787 -1.128131 -1.706536   \n",
       "4  7519.0  1.234235  3.019740 -4.304597  4.732795  3.624201 -1.357746   \n",
       "\n",
       "         V7        V8        V9  ...       V21       V22       V23       V24  \\\n",
       "0 -2.537387  1.391657 -2.770089  ...  0.517232 -0.035049 -0.465211  0.320198   \n",
       "1  0.325574 -0.067794 -0.270953  ...  0.661696  0.435477  1.375966 -0.293803   \n",
       "2  0.562320 -0.399147 -0.238253  ... -0.294166 -0.932391  0.172726 -0.087330   \n",
       "3 -3.496197 -0.248778 -0.247768  ...  0.573574  0.176968 -0.436207 -0.053502   \n",
       "4  1.713445 -0.496358 -1.282858  ... -0.379068 -0.704181 -0.656805 -1.632653   \n",
       "\n",
       "        V25       V26       V27       V28  Amount  Class  \n",
       "0  0.044519  0.177840  0.261145 -0.143276    0.00      1  \n",
       "1  0.279798 -0.145362 -0.252773  0.035764  529.00      1  \n",
       "2 -0.156114 -0.542628  0.039566 -0.153029  239.93      1  \n",
       "3  0.252405 -0.657488 -0.827136  0.849573   59.00      1  \n",
       "4  1.488901  0.566797 -0.010016  0.146793    1.00      1  \n",
       "\n",
       "[5 rows x 31 columns]"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.read_csv(\"creditcard2.csv\")\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4b44a1a",
   "metadata": {},
   "source": [
    "数据标签分布"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "e7357bee",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T08:53:00.915377Z",
     "start_time": "2023-05-26T08:53:00.896520Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    492\n",
       "1    492\n",
       "Name: Class, dtype: int64"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.value_counts(data['Class'], sort = True).sort_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "672b9f2c",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T08:53:01.423122Z",
     "start_time": "2023-05-26T08:53:01.413263Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Time', 'V1', 'V2', 'V3', 'V4', 'V5', 'V6', 'V7', 'V8', 'V9', 'V10',\n",
       "       'V11', 'V12', 'V13', 'V14', 'V15', 'V16', 'V17', 'V18', 'V19', 'V20',\n",
       "       'V21', 'V22', 'V23', 'V24', 'V25', 'V26', 'V27', 'V28', 'Amount',\n",
       "       'Class'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71d86e82",
   "metadata": {},
   "source": [
    "数据标准化处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "37e448a6",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T08:53:02.419263Z",
     "start_time": "2023-05-26T08:53:02.393434Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "      <th>V5</th>\n",
       "      <th>V6</th>\n",
       "      <th>V7</th>\n",
       "      <th>V8</th>\n",
       "      <th>V9</th>\n",
       "      <th>V10</th>\n",
       "      <th>...</th>\n",
       "      <th>V21</th>\n",
       "      <th>V22</th>\n",
       "      <th>V23</th>\n",
       "      <th>V24</th>\n",
       "      <th>V25</th>\n",
       "      <th>V26</th>\n",
       "      <th>V27</th>\n",
       "      <th>V28</th>\n",
       "      <th>Class</th>\n",
       "      <th>normAmount</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-2.312227</td>\n",
       "      <td>1.951992</td>\n",
       "      <td>-1.609851</td>\n",
       "      <td>3.997906</td>\n",
       "      <td>-0.522188</td>\n",
       "      <td>-1.426545</td>\n",
       "      <td>-2.537387</td>\n",
       "      <td>1.391657</td>\n",
       "      <td>-2.770089</td>\n",
       "      <td>-2.772272</td>\n",
       "      <td>...</td>\n",
       "      <td>0.517232</td>\n",
       "      <td>-0.035049</td>\n",
       "      <td>-0.465211</td>\n",
       "      <td>0.320198</td>\n",
       "      <td>0.044519</td>\n",
       "      <td>0.177840</td>\n",
       "      <td>0.261145</td>\n",
       "      <td>-0.143276</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-3.043541</td>\n",
       "      <td>-3.157307</td>\n",
       "      <td>1.088463</td>\n",
       "      <td>2.288644</td>\n",
       "      <td>1.359805</td>\n",
       "      <td>-1.064823</td>\n",
       "      <td>0.325574</td>\n",
       "      <td>-0.067794</td>\n",
       "      <td>-0.270953</td>\n",
       "      <td>-0.838587</td>\n",
       "      <td>...</td>\n",
       "      <td>0.661696</td>\n",
       "      <td>0.435477</td>\n",
       "      <td>1.375966</td>\n",
       "      <td>-0.293803</td>\n",
       "      <td>0.279798</td>\n",
       "      <td>-0.145362</td>\n",
       "      <td>-0.252773</td>\n",
       "      <td>0.035764</td>\n",
       "      <td>1</td>\n",
       "      <td>0.248839</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-2.303350</td>\n",
       "      <td>1.759247</td>\n",
       "      <td>-0.359745</td>\n",
       "      <td>2.330243</td>\n",
       "      <td>-0.821628</td>\n",
       "      <td>-0.075788</td>\n",
       "      <td>0.562320</td>\n",
       "      <td>-0.399147</td>\n",
       "      <td>-0.238253</td>\n",
       "      <td>-1.525412</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.294166</td>\n",
       "      <td>-0.932391</td>\n",
       "      <td>0.172726</td>\n",
       "      <td>-0.087330</td>\n",
       "      <td>-0.156114</td>\n",
       "      <td>-0.542628</td>\n",
       "      <td>0.039566</td>\n",
       "      <td>-0.153029</td>\n",
       "      <td>1</td>\n",
       "      <td>0.112862</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-4.397974</td>\n",
       "      <td>1.358367</td>\n",
       "      <td>-2.592844</td>\n",
       "      <td>2.679787</td>\n",
       "      <td>-1.128131</td>\n",
       "      <td>-1.706536</td>\n",
       "      <td>-3.496197</td>\n",
       "      <td>-0.248778</td>\n",
       "      <td>-0.247768</td>\n",
       "      <td>-4.801637</td>\n",
       "      <td>...</td>\n",
       "      <td>0.573574</td>\n",
       "      <td>0.176968</td>\n",
       "      <td>-0.436207</td>\n",
       "      <td>-0.053502</td>\n",
       "      <td>0.252405</td>\n",
       "      <td>-0.657488</td>\n",
       "      <td>-0.827136</td>\n",
       "      <td>0.849573</td>\n",
       "      <td>1</td>\n",
       "      <td>0.027753</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.234235</td>\n",
       "      <td>3.019740</td>\n",
       "      <td>-4.304597</td>\n",
       "      <td>4.732795</td>\n",
       "      <td>3.624201</td>\n",
       "      <td>-1.357746</td>\n",
       "      <td>1.713445</td>\n",
       "      <td>-0.496358</td>\n",
       "      <td>-1.282858</td>\n",
       "      <td>-2.447469</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.379068</td>\n",
       "      <td>-0.704181</td>\n",
       "      <td>-0.656805</td>\n",
       "      <td>-1.632653</td>\n",
       "      <td>1.488901</td>\n",
       "      <td>0.566797</td>\n",
       "      <td>-0.010016</td>\n",
       "      <td>0.146793</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000470</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 30 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         V1        V2        V3        V4        V5        V6        V7  \\\n",
       "0 -2.312227  1.951992 -1.609851  3.997906 -0.522188 -1.426545 -2.537387   \n",
       "1 -3.043541 -3.157307  1.088463  2.288644  1.359805 -1.064823  0.325574   \n",
       "2 -2.303350  1.759247 -0.359745  2.330243 -0.821628 -0.075788  0.562320   \n",
       "3 -4.397974  1.358367 -2.592844  2.679787 -1.128131 -1.706536 -3.496197   \n",
       "4  1.234235  3.019740 -4.304597  4.732795  3.624201 -1.357746  1.713445   \n",
       "\n",
       "         V8        V9       V10  ...       V21       V22       V23       V24  \\\n",
       "0  1.391657 -2.770089 -2.772272  ...  0.517232 -0.035049 -0.465211  0.320198   \n",
       "1 -0.067794 -0.270953 -0.838587  ...  0.661696  0.435477  1.375966 -0.293803   \n",
       "2 -0.399147 -0.238253 -1.525412  ... -0.294166 -0.932391  0.172726 -0.087330   \n",
       "3 -0.248778 -0.247768 -4.801637  ...  0.573574  0.176968 -0.436207 -0.053502   \n",
       "4 -0.496358 -1.282858 -2.447469  ... -0.379068 -0.704181 -0.656805 -1.632653   \n",
       "\n",
       "        V25       V26       V27       V28  Class  normAmount  \n",
       "0  0.044519  0.177840  0.261145 -0.143276      1    0.000000  \n",
       "1  0.279798 -0.145362 -0.252773  0.035764      1    0.248839  \n",
       "2 -0.156114 -0.542628  0.039566 -0.153029      1    0.112862  \n",
       "3  0.252405 -0.657488 -0.827136  0.849573      1    0.027753  \n",
       "4  1.488901  0.566797 -0.010016  0.146793      1    0.000470  \n",
       "\n",
       "[5 rows x 30 columns]"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler\n",
    "data['normAmount'] = MinMaxScaler().fit_transform(data['Amount'].values.reshape(-1, 1))\n",
    "data = data.drop(['Time','Amount'],axis=1)\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29926b52",
   "metadata": {},
   "source": [
    "数据集划分"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "33cbb96a",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T08:53:03.311440Z",
     "start_time": "2023-05-26T08:53:03.302108Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "7342eed4",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T08:53:03.707935Z",
     "start_time": "2023-05-26T08:53:03.689359Z"
    }
   },
   "outputs": [],
   "source": [
    "X = data.iloc[:, data.columns != 'Class'].values\n",
    "y = data.iloc[:, data.columns == 'Class']['Class'].values\n",
    "\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X,y,test_size = 0.3, random_state = 114)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3bf7b7e9",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-24T08:08:34.548450Z",
     "start_time": "2023-05-24T08:08:34.537494Z"
    }
   },
   "source": [
    "逻辑回归模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "71a9a4ca",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T08:53:04.543362Z",
     "start_time": "2023-05-26T08:53:04.538843Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.model_selection import KFold, cross_validate\n",
    "from sklearn.metrics import recall_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "8096e302",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T08:53:09.140779Z",
     "start_time": "2023-05-26T08:53:05.107886Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-------------------------------------------\n",
      "正则化惩罚力度:  0.01\n",
      "-------------------------------------------\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n",
      "[Parallel(n_jobs=-1)]: Done   2 out of   5 | elapsed:    2.6s remaining:    3.8s\n",
      "[Parallel(n_jobs=-1)]: Done   5 out of   5 | elapsed:    2.6s finished\n",
      "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n",
      "[Parallel(n_jobs=-1)]: Done   2 out of   5 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.91428571 0.9137931  0.82089552 0.90140845 0.86567164]\n",
      "-------------------------------------------\n",
      "正则化惩罚力度:  0.1\n",
      "-------------------------------------------\n",
      "\n",
      "[0.91428571 0.93103448 0.89552239 0.92957746 0.91044776]\n",
      "-------------------------------------------\n",
      "正则化惩罚力度:  1\n",
      "-------------------------------------------\n",
      "\n",
      "[0.91428571 0.94827586 0.94029851 0.91549296 0.92537313]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done   5 out of   5 | elapsed:    1.4s finished\n",
      "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n",
      "[Parallel(n_jobs=-1)]: Done   2 out of   5 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=-1)]: Done   5 out of   5 | elapsed:    0.0s finished\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>c_param</th>\n",
       "      <th>recall</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.01</td>\n",
       "      <td>0.883211</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.10</td>\n",
       "      <td>0.916174</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.00</td>\n",
       "      <td>0.928745</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   c_param    recall\n",
       "0     0.01  0.883211\n",
       "1     0.10  0.916174\n",
       "2     1.00  0.928745"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def kflod_scores(X,y):\n",
    "    fold = KFold(5,shuffle=False)\n",
    "    c_param_range = [0.01, 0.1, 1]\n",
    "    show_result = pd.DataFrame()\n",
    "    recalls = []\n",
    "    cv = KFold(n_splits=5, shuffle=True, random_state=114)\n",
    "    for c_param in c_param_range:\n",
    "        lr_model = LogisticRegression(C=c_param, penalty=\"l2\")\n",
    "        print('-------------------------------------------')\n",
    "        print('正则化惩罚力度: ', c_param)\n",
    "        print('-------------------------------------------')\n",
    "        print('')\n",
    "        result = cross_validate(\n",
    "            lr_model\n",
    "            , X\n",
    "            , y\n",
    "            , cv=cv\n",
    "            , scoring=\"recall\"\n",
    "            , verbose=True\n",
    "            , n_jobs=-1\n",
    "        )\n",
    "        print(result[\"test_score\"])\n",
    "        recalls.append(np.mean(result[\"test_score\"]))\n",
    "\n",
    "    show_result[\"c_param\"] = list(c_param_range)\n",
    "    show_result[\"recall\"] = recalls\n",
    "    return show_result\n",
    "kflod_scores(X_train,y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0bf84048",
   "metadata": {},
   "source": [
    "验证模型结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "99500e3d",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T08:53:18.156570Z",
     "start_time": "2023-05-26T08:53:16.671043Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9182389937106918"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr = LogisticRegression(C = 100, penalty = 'l2')\n",
    "lr.fit(X_train,y_train)\n",
    "yhat = lr.predict(X_test)\n",
    "recall_score_value = recall_score(y_test, yhat)\n",
    "recall_score_value"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5068ca4",
   "metadata": {},
   "source": [
    "刚刚进行的建模存在一些问题\n",
    "<br>\n",
    "(1).过程不够严谨，诸如测试集中测试结果不能指导建模、参数选取及搜索区间选取没有理论依据等问题仍然存在；<br>\n",
    "(2).执行效率太低，如果面对更多的参数（这是更一般的情况），手动执行过程效率太低，无法进行超大规模的参数挑选；<br>\n",
    "(3).结果不够精确，一次建模结果本身可信度其实并不高，我们很难证明上述挑选出来的参数就一定在未来数据预测中拥有较高准确率。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7105be6",
   "metadata": {},
   "source": [
    "### 网格搜索"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "834d8837",
   "metadata": {},
   "source": [
    "本节我们主要介绍sklearn中调参的核心工具——GridSearchCV"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "553c6535",
   "metadata": {},
   "source": [
    "- 参数与超参数\n",
    "- 参数的数值计算由一整套数学过程决定，在选定方法后，其计算过程基本不需要人工参与\n",
    "- 超参数，是模型在构建过程中，无法通过一个严谨的数学流程给出最优解的参数，因此需要人工参与进行调节。\n",
    "\n",
    "sklearn中逻辑回归模型中，就包含以下参数\n",
    "\n",
    " <img src=\"./images/18.jpg\" alt=\"1\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e7cf4c1",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-24T09:30:33.914268Z",
     "start_time": "2023-05-24T09:30:33.903823Z"
    }
   },
   "source": [
    "### sklearn中GridSearchCV的使用方法"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65f9eae9",
   "metadata": {},
   "source": [
    "sklearn中最常用的搜索策略就是使用GridSearchCV进行全搜索，即对参数空间内的所有参数进行搜索\n",
    " <img src=\"./images/19.jpg\" alt=\"1\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fcfa90a6",
   "metadata": {},
   "source": [
    "该方法在model_selection模块下，同样也是以评估器形式存在，我们可以通过如下方式进行导入："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "77d7f96a",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T09:58:47.756306Z",
     "start_time": "2023-05-26T09:58:47.750457Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import GridSearchCV"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c75d82d",
   "metadata": {},
   "source": [
    "GridSearchCV它的参数主要如下\n",
    "\n",
    " <img src=\"./images/20.jpg\" alt=\"1\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ada37207",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-24T09:36:13.314984Z",
     "start_time": "2023-05-24T09:36:13.297119Z"
    }
   },
   "source": [
    "通过代码来查看一下它具体的用法，首先导入数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "cd7af0bc",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T09:54:53.689950Z",
     "start_time": "2023-05-26T09:54:53.534427Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_iris\n",
    "X, y = load_iris(return_X_y=True)\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=114)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "10dfb1b8",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T09:55:13.082242Z",
     "start_time": "2023-05-26T09:55:13.071883Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(112, 4)"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f8e83d6",
   "metadata": {},
   "source": [
    "创建模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "6002e844",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T09:55:45.637798Z",
     "start_time": "2023-05-26T09:55:45.621929Z"
    }
   },
   "outputs": [],
   "source": [
    "lgr = LogisticRegression(max_iter=int(1e6), solver='saga')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "6057522f",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T09:55:57.071457Z",
     "start_time": "2023-05-26T09:55:57.057458Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'C': 1.0,\n",
       " 'class_weight': None,\n",
       " 'dual': False,\n",
       " 'fit_intercept': True,\n",
       " 'intercept_scaling': 1,\n",
       " 'l1_ratio': None,\n",
       " 'max_iter': 1000000,\n",
       " 'multi_class': 'auto',\n",
       " 'n_jobs': None,\n",
       " 'penalty': 'l2',\n",
       " 'random_state': None,\n",
       " 'solver': 'saga',\n",
       " 'tol': 0.0001,\n",
       " 'verbose': 0,\n",
       " 'warm_start': False}"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lgr.get_params()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31c96a00",
   "metadata": {},
   "source": [
    "创建参数空间"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "d2e49710",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T09:57:47.859164Z",
     "start_time": "2023-05-26T09:57:47.850718Z"
    }
   },
   "outputs": [],
   "source": [
    "param_grid_simple = {'penalty': ['l1', 'l2'],\n",
    "                     'C': [1, 0.5, 0.1, 0.05, 0.01]}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "f72ce2b2",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T09:59:39.160803Z",
     "start_time": "2023-05-26T09:59:39.144748Z"
    }
   },
   "outputs": [],
   "source": [
    "GridSearchCV?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c34bb88d",
   "metadata": {},
   "source": [
    "实例化网格搜索评估器"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "91c17210",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T09:58:49.781964Z",
     "start_time": "2023-05-26T09:58:49.773833Z"
    }
   },
   "outputs": [],
   "source": [
    "search = GridSearchCV(estimator=lgr,\n",
    "                      param_grid=param_grid_simple)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7fbbae42",
   "metadata": {},
   "source": [
    "训练网格搜索评估器"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "1d84f5fb",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T09:59:11.627790Z",
     "start_time": "2023-05-26T09:59:10.587695Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-7 {color: black;background-color: white;}#sk-container-id-7 pre{padding: 0;}#sk-container-id-7 div.sk-toggleable {background-color: white;}#sk-container-id-7 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-7 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-7 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-7 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-7 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-7 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-7 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-7 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-7 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-7 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-7 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-7 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-7 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-7 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-7 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-7 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-7 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-7 div.sk-item {position: relative;z-index: 1;}#sk-container-id-7 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-7 div.sk-item::before, #sk-container-id-7 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-7 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-7 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-7 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-7 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-7 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-7 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-7 div.sk-label-container {text-align: center;}#sk-container-id-7 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-7 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-7\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>GridSearchCV(estimator=LogisticRegression(max_iter=1000000, solver=&#x27;saga&#x27;),\n",
       "             param_grid={&#x27;C&#x27;: [1, 0.5, 0.1, 0.05, 0.01],\n",
       "                         &#x27;penalty&#x27;: [&#x27;l1&#x27;, &#x27;l2&#x27;]})</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-7\" type=\"checkbox\" ><label for=\"sk-estimator-id-7\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">GridSearchCV</label><div class=\"sk-toggleable__content\"><pre>GridSearchCV(estimator=LogisticRegression(max_iter=1000000, solver=&#x27;saga&#x27;),\n",
       "             param_grid={&#x27;C&#x27;: [1, 0.5, 0.1, 0.05, 0.01],\n",
       "                         &#x27;penalty&#x27;: [&#x27;l1&#x27;, &#x27;l2&#x27;]})</pre></div></div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-8\" type=\"checkbox\" ><label for=\"sk-estimator-id-8\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">estimator: LogisticRegression</label><div class=\"sk-toggleable__content\"><pre>LogisticRegression(max_iter=1000000, solver=&#x27;saga&#x27;)</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-9\" type=\"checkbox\" ><label for=\"sk-estimator-id-9\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LogisticRegression</label><div class=\"sk-toggleable__content\"><pre>LogisticRegression(max_iter=1000000, solver=&#x27;saga&#x27;)</pre></div></div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "GridSearchCV(estimator=LogisticRegression(max_iter=1000000, solver='saga'),\n",
       "             param_grid={'C': [1, 0.5, 0.1, 0.05, 0.01],\n",
       "                         'penalty': ['l1', 'l2']})"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da217281",
   "metadata": {},
   "source": [
    "结果查看，可以通过以下属性进行结果查看\n",
    "\n",
    " <img src=\"./images/21.jpg\" alt=\"1\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ea58d5f",
   "metadata": {},
   "source": [
    "best_estimator_：训练完成后的最佳评估器"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "id": "f56458ed",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T10:00:38.996858Z",
     "start_time": "2023-05-26T10:00:38.982223Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-8 {color: black;background-color: white;}#sk-container-id-8 pre{padding: 0;}#sk-container-id-8 div.sk-toggleable {background-color: white;}#sk-container-id-8 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-8 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-8 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-8 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-8 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-8 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-8 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-8 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-8 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-8 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-8 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-8 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-8 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-8 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-8 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-8 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-8 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-8 div.sk-item {position: relative;z-index: 1;}#sk-container-id-8 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-8 div.sk-item::before, #sk-container-id-8 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-8 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-8 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-8 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-8 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-8 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-8 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-8 div.sk-label-container {text-align: center;}#sk-container-id-8 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-8 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-8\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LogisticRegression(C=1, max_iter=1000000, solver=&#x27;saga&#x27;)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-10\" type=\"checkbox\" checked><label for=\"sk-estimator-id-10\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LogisticRegression</label><div class=\"sk-toggleable__content\"><pre>LogisticRegression(C=1, max_iter=1000000, solver=&#x27;saga&#x27;)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "LogisticRegression(C=1, max_iter=1000000, solver='saga')"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search.best_estimator_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dfa7d058",
   "metadata": {},
   "source": [
    "可以调用逻辑回归评估器的所有属性："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "d992d4bf",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T10:01:45.122726Z",
     "start_time": "2023-05-26T10:01:45.111051Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.00573706,  1.11175793, -2.24681269, -1.0125985 ],\n",
       "       [ 0.18785594, -0.41654231, -0.17209   , -0.55342379],\n",
       "       [-0.18211888, -0.69521562,  2.41890269,  1.56602229]])"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search.best_estimator_.coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "618dbcaf",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T10:02:46.547454Z",
     "start_time": "2023-05-26T10:02:46.540548Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.9821428571428571, 0.9473684210526315)"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 查看训练误差、测试误差\n",
    "search.best_estimator_.score(X_train,y_train), search.best_estimator_.score(X_test,y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "id": "ae3c3f4e",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-24T09:43:26.810165Z",
     "start_time": "2023-05-24T09:43:26.789303Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'C': 1,\n",
       " 'class_weight': None,\n",
       " 'dual': False,\n",
       " 'fit_intercept': True,\n",
       " 'intercept_scaling': 1,\n",
       " 'l1_ratio': None,\n",
       " 'max_iter': 1000000,\n",
       " 'multi_class': 'auto',\n",
       " 'n_jobs': None,\n",
       " 'penalty': 'l2',\n",
       " 'random_state': None,\n",
       " 'solver': 'saga',\n",
       " 'tol': 0.0001,\n",
       " 'verbose': 0,\n",
       " 'warm_start': False}"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 查看参数\n",
    "search.best_estimator_.get_params()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ef65814",
   "metadata": {},
   "source": [
    "best_score_：最优参数时交叉验证平均得分"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "f5afa5bd",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-26T10:01:06.874222Z",
     "start_time": "2023-05-26T10:01:06.861811Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9727272727272727"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search.best_score_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a655a5cb",
   "metadata": {},
   "source": [
    "&emsp;&emsp;在默认情况下（未修改网格搜索评估器中评估指标参数时），此处的score就是准确率。此处有两点需要注意：      \n",
    "&emsp;&emsp;其一：该指标和训练集上整体准确率不同，该指标是交叉验证时验证集准确率的平均值，而不是所有数据的准确率；      \n",
    "&emsp;&emsp;其二：该指标是网格搜索在进行参数挑选时的参照依据。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "id": "d1158087",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-24T09:44:01.455389Z",
     "start_time": "2023-05-24T09:44:01.441106Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'C': 1, 'penalty': 'l2'}"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ab67bfb5",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
